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ABSTRACT 


Herein  is  presented  a  Kolmogorov-Smirnov  test  for  grouped 
samples.   It  constitutes  an  application  of  W.J.  Conover's 
procedure,  which  was  originally  designed  for  calculating 
exact  critical  levels  for  test  with  discrete  distributions. 
This  test  extends  the  goodness-of-f it  test  to  samples  which 
are  grouped  into  intervals,   Critical  levels  in  the  two-sided 
case  were  calculated  to  a  close  approximation. 

Some  examples  of  the  application  of  this  extension  of 
the  Kolmogorov-Smirnov  test  are  included  and  comparisons 
with  the  Chi-square  test,  the  Kolmogorov-Smirnov  test  for 
continuous  distribution  and  for  grouped  camples  are  made. 
Comparisons  among  the  three  goodness-of-f it  tests  based  on 
the  results  of  computer  simulations  are  made. 
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I.   INTRODUCTION 

One  of  the  most  important  tests  in  statistical 
applications  arises  in  testing  hypotheses  about  the 
distribution  of  a  population.   Before  specifying  a  model, 
usually  we  should  look  at  the  data  to  see  if  they  appear 
to  have  come  from  the  distribution  which  we  expect  to  use 
for  the  model.   This  can  be  approached  through  the  histogram, 
which  gives  us  information  about  the  density  function  of 
the  underlying  distribution.   Another  approach  is  the  sample 
distribution  function,  which  gives  us  an  estimate  of  the 
underlying  cumulative  distribution  function. 

We  call  a  test  "a  test  of  goodness-of-f it"  if  the  test 
is  concerned  with  the  agreement  between  the  distribution 
of  a  set  of  sample  values  and  some  theoretical  distribution, 

Much  work  has  been  devoted  to  finding  test  statistics 
whose  distributions  do  not  depend  on  parameters  in  the 
distribution  of  the  underlying  population.   Such  tests  are 
commonly  called  distribution-free  tests. 

One  of  the  most  well-known  and  useful  goodness-of-f it 
tests  is  the  Kolmogorov-Smirnov  Goodness-of-f it  tests  , 
The  Kolmogorov-Smirnov  test  treats  individual  observations 
separately  and  thus  does  not  lose  information  through 
grouping  as  does  the  Chi-square  test.   Consequently,  in 
the  continuous  case,  the  Chi-square  test  is  frequently  less 
powerful  than  the  Kolmogorov-Smirnov  test. 


It  is  also  known  that  the  Kolmogorov-Smirnov  test  is 
conservative  when  the  hypothesized  distribution  function 
is  not  continuous.   In  many  situations,  observations  from 
a  continuous  distribution  are  grouped.   However,  studies 
of  the  modifications  of  the  Kolmogorov-Smirnov  test  for 
use  with  grouped  data  appear  to  be  rather  limited. 

The  Chi-square  test,  suggested  by  Pearson  (1900),  is 
well  suited  for  use  with  grouped  data,  whereas  the  K-S 
test  is  for  random  samples  from  continuous  populations. 
W.J.  Conover's  procedure  was  designed  to  calculate  critical 
levels  for  random  samples  from  discrete  distributions.   Since 
grouping  the  data  from  continuous  or  discrete  populations 
will  result  in  a  corresponding  underlying  distribution 
which  is  discrete,  Conover's  procedure  can  also  be  used 
for  grouped  data.   The  general  problem  of  defining  the 
classes  or  determining  the  class  boundaries  in  some  optimal 
way  has  apparently  received  limited  attention,   In  what 
follows  we  propose  a  Kolmogorov-Smirnov  test  for  grouped 
data  which  shows  a  method  of  finding  (almost)  exact 
critical  levels  and  the  power  of  this  test.   Thus  the 
Kolmogorov-Smirnov  test  may  be  used  as  a  goodness-of-f it 
test,  regardless  of  whether  the  hypothesized  distribution 
is  continuous,  or  whether  the  samples  are  grouped. 

The  following  listing  constitutes  the  description  or 
definition  of  notation  used  herein: 


Notation  Description 

K-S  test  Kolmogorov-Smirnov  test  for 

continuous  distribution  function 
and  ungrouped  data. 

K-S.  .  test  Kolmogorov-Smirnov  test  for 

^g'  grouped  data. 

S  Empirical  distribution  function 

of  a  random  sample  of  size  n 

5  Empirical  distribution  function 

of  a  random  sample  of  size  n 
which  is  grouped. 

n  Sample  size. 

a  Critical  level  of  test 

6  Critical  value  associated  with 

continuous  distribution  function 
and  ungrouped  data, 

6  Critical  value  associated  with 

g  grouped  data. 

F  Hypothesized  distribution 

function . 

F  Some  population  distribution 

function 

b.  Some  fixed  real  number  represent- 

ing the  class  boundary, 

H  Null  hypothesis 

Hx  Alternative  hypothesis 

p.  Relative  frequency  of  acceptance 

1  of  null  hypothesis. 

X(On   (2)^~     (n)   Ordered  observed  random 

sample  from  distribution  F 
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II.   GENERAL  DESCRIPTION  OF  THE  TEST 

The  Kolmogorov-Smirnov  test  for  goodness-of-f it  is 

based  on  the  maximum  vertical  difference  between  the 

empirical  and  the  hypothesized  cumulative  distribution 

function.   Under  H  ,  the  empirical  distribution  function 

o 

is  expected  to  be  close  to  the  specified  distribution 

function,  F  (•).   If  the  maximum  vertical  difference 
'   o 

between  the  specified  and  the  empirical  distribution  is 
not  small  enough  for  some  x,  say  greater  than  a  critical 
value  6 ,  this  may  be  considered  evidence  that  the  hypothe- 
sized distribution  is  not  the  one  from  which  the  sample 
was  drawn. 

Suppose  F  ( • )  is  the  hypothesized  distribution  and 
S  (•)  is  the  empirical  distribution  function,  then 

k  th 

S   (x)  =  —     for  x  between  the  k —  and  (k  +  1)  st 
n       n 

largest  values  in  the  sample  and  n  is 
the  sample  size. 
The  Kolmogorov-Smirnov  test  is:   accept 
the  hypothesis  H  :   X  ~-  F  (•  )   if  and  only  if 


D   =  Sup 
n      ^ 


F  (x)  -  S  (x) 

ov  '     nv  ' 


^  6  ,   where 


—  oo  <x<°° 

6  is  adjusted  to  give  a  level  a  test.   This  is  sometimes 

called  the  two-sided  Kolmogorov-Smirnov  statistic  as 

opposed  to  the  one-sided  statistics  D   and  D   ,  where 

n       n  ' 


Dq+   =    Sup    f  Sn(x)    -    Fq(x) 


-oo<X<°° 


{'. 


Dn      =    Sup    <(  F    (x)    -    Sn(x) 


00<X<00 


It    is  well   know  that    if   X.,    X, , ,    X      is   a  random 

1  '   2      >   n 

sample  from  a  continuous  distubition  function  F  (•),  D 

o    '   n 

is  distribution-free   i.e.;  has  distribution  independent 

of  F  (• )• 
ov 

Let  Xj ,  X2 , ,  X   be  independent  random  variables 

with  the  common  cumulative  distribution  function  F  (•). 

ov  ' 


Furthermore  let  x,    .         x,    N 

(  i )     (2) 


x,    .    be  the  result  of 
(n) 


n  independent  observations,  arranged  in  order  size,  that 
is,  the  n  order  statistics.   Suppose  the  above  random 
samples  are  divided  into  k  groups,  given  by  class  boundaries 


(J). 


x.  ,  i=l,  2,  k, where  the  b-'s  are  the  outcomes 

j  jj  «j 

(observed  values)  of  certain  of  the  order  statistic  X 

or 

Let  Sto(x)  be  the  sample  distribution  function  based  on 

cr 

the  grouped  data.   Then  S  (x)  is  a  step  function  which 
jumps  at  class  boundaries  and  so  can  be  written; 


Sg(x) 

nv  J 


=  < 


o 


Ji 


n 
1 


for  x  <  X 


(1) 


for  X,  .  N  <:  x  <  X,  .    x 
(Ji>         <Ji+l> 


for  x  >  X 


(n) 
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For  the  grouped  data,  the  maximum  vertical  difference 

between  F  (x)  and  Sto(x)  occurs  at  one  of  the  class  bound- 
o         n 

aries.   The  two-sided  Kolmogorov-Smirnov  test  statistics 
for  grouped  data  (K  -  S,  *  statistic)  is  defined   to  be; 


D  =  max 
n 


F  (x)  -  Sg(x) 
ov  '     nv  ' 


X£  \   X  . X 


k-'l 


The  "one-sided  K  -  S,  N  test"  statistics  are; 

(g) 


D    =  max 
n 


Sg(x)  -  F  fx) 
n       o 


xz<x  . x 


k- 


D 


=  max 


xe  <x  .  '—•'-x 


<x  . 
I  3X 


1 


{'. 


(x)  - 


Sg(x) 

nv  ' 


In  what  follows,  it  will  be  shown  that  these  three 
statictics  are  distribution  free,  provided  that  F  (x)  is 
continuous. 

Make  the  change  of  variables; 


y  =  F  (x  .  ) 

Y.  =  F  (X.) 
3  ov  j' 


,    Since  F  (x .  )  is 


nondecreasing,  the  inequality  X.  •$  x.   is  equivalent  to 

sg.    *     x  -  1 


F  (x.)  ,<  F  (x.  )  or  Y.  <  y.   Thus  S&(x.  )  = 
ov  r  v   ov  j  .       1   *         nv  j . 


=  —  times 
n    n 


(the  number  of  X,  ,<  x.  )  =  -  (the  number  of  Y.  <  y)  =  H(y) 


J  v   J 


Furthermore,  the  distribution  of  Y.  is; 

J 


pfY^y"  =  P  [x±  v<  x.  =  Fo(Xj  )  =  y,   Thus, 
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Y.  has  a  uniform  distribution  and  D.  can  be  written  as 
3  n 


D   =  max 
n 


xe 


{x.  —x.  ) 
\     3,  Jk_2) 


F  (x)  -  Sg(x) 
o   '     nv  y 


=  max 


F  (x .  )  -  Sg(x  .  ) 


o-  J 


nv  J 


i=l,  2— k 


so  that , 


F  (x.  )  -  Sg(x.  ) 
oK    3i  nv  J-i 


y  -  H(y) 


and  D  =  max 
n 


xe|  x. x  .  \ 


F  (x)  -  SB(x) 
o        n      ' 


=   max 


y  -  H(y)| 


The  expression  on  the  right  is  the  vertical  distance  between 

the  sample  distribution  of  Y  ,  Y   Y   each  U  (0,1). 

Since  this  expression  does  not  depend  upon  F  ,  D   is  distri- 
bution free.   In  this  case,  where  the  class  boundaries  are 
order  statistic,  D   is  distribution  free.  Massey ' s  procedure  (8 ) 
or  Davidson's  method  (4)  could  be  used  to  calculate  critical 
values  for  this  test. 

In  the  case  where  the  class  boundaries  are  fixed 

constant,  D   is  no  longer  distribution  free.   It  can  be 
n 

described  as  follows;   Suppose  the  class  boundaries  are 

fixed  constants  in  advance  say  b..   The  value  of  F  (•)  at 

'   l  o 

b.  is  F  (b.)-   As  before,  make  the  change  of  "variable" 

1       o   1 

y  =  F  (b.).   However,  since  b.  is  constant,  y  =  F  (b.) 
o   1  1  oi 

cannot  be  transformed  to  Y  =  F  (B.).   In  other  words  D   = 

o   i  n 
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max    .IF  (x)  -  Sg(x) 
o       n v 


xe{b.) 


max 
i 


F  (b. )  -  Sg(b. ) 


=f=     max 


y  -  H(y) 


Therefore,  it  is  suggested  that 


Conover's  procedure  be  used  to  calculate  critical  levels 

for  K  -  S,  N  when  the  class  boundaries  are  fixed  constant 
(g) 
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III.   TEST  PROCEDURE  AND  CRITICAL  LEVEL 

A ,   TEST-PROCEDURE 

Unlike  the  Chi-square  test  and  the  Kolmogorov- 
Smirnov  test,  in  which  critical  values  are  calculated 
which  correspond  to  selected  critical  levels,  with  Conover's 
procedure  one  calculates  critical  levels  which  correspond 
to  selected  critical  values,   The  procedure  is  as  follows, 

Let  X. ,  X2 X   be  a  random  sample  of  size  n,  drawn  from 

some  unknown  discrete  population  distribution  F  (•)• 

A 

The  hypothesis  is:   H:F(#)=F(«)  where  F  (•)  has  all 

J  *  ox       o  o 

parameters  specified,   The  alternative  is  H  :  F  (•)  =f= 

F  (•).   The  test  statistic,  either  D  ,  D   ,  or  D 
o v  '  .  n    n  '      n 

depending  on  which  is  desired,  is  calculated.   Further,  the 
critical  level  is  computed  and  if  the  value  of  this  critical 
level  is  greater  than  or  equal  to  that  specified  in  advance 
(usually  either  005  or  001),  the  null  hypothesis  is  accepted; 
otherwise  it  is  rejected. 

Let  6  be  some  fixed  real  number,  o  <  6  <  1.   The  critical 
value  of  the  test,  that  is  P  [~D   £.  6 1  for  the  two-sided 
Kolmogorov-Smirnov  (discrete)  test,  can  be  computed  by  using 
a  procedure  due  to  W.J.  Conover  (2),   Infact  this  is  an  • 
approximation  method,  since  P  |~D  >,   6  1  is  obtained  by 

calculating  P  I"  D  ~  ^  6  1  and  P  I  D    ^  6~]  ,  where  6  =  max 

-   +       -    +  -       + 

(6,6)  and  6  ,   6   are  the  observed  values  of  D   and  D 

respectively.     P   D   >  6l    is  taken  to  be  approximately 
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approximately        P  |D  ~   ^  6  1  +  P   j"Dn+    fc.  6  1  .      To   see   the 
adequacy   of    this   approximation,    we  proceed   as    follows; 

P   [Dn    *   6]  =   P  [(Dn+    '  6)    b>    (Dn~    *  6)] 
-   P    [Dn+>,   6]    +   P  [Dn~      ^   6]    -p    [    (Dn+    =>  6>     °  (Dn      *  ~6) 


p  [(Dn+  >,  M    nr(Dn-  >,  6  ]=  p  [dr+  >,  6_ 


D         >6   D      >.  6 

n       '    '    n    ^ 


••(1) 


Since      pTd        >>  6  I  Dn+   >    6]   ^    p   [Dn      ^  6]        equation    (1) 


becomes : 


n 


(D         :>    6)     n  (D         >    6) 
v    n       '      '         v    n       '      y 


<      P 


+ 

~ 

D         >    6 

P 

D 

Z   6 

n 

n 

/ 

an    approximate   value   of   P     D      >.  6        is   thus 


The   error 


P  [~D      ^6]^PrD    +    ^6l+pfD         £.  <5  1      , 
Ln^j  l.    n  J  I   n      ^      J 

of   this   approximation    is   P      (D        >,  6 )  n  (D  >,  6 ) 

Lch    is    less   than   the   product    p|~D        >,6]pfD~>,6] 


whi 


But  in  practice,  6  is  taken  so  that  P 


D  +  >y  6 

n  ' 


} 


and 


K 


>.  6 


are  small  and  then  the  product  is  quite  small 


and  can  be  safely  ignored.   For  example,  suppose  both  one- 
sided tests  have  the  same  critical  level,  say,  0.05,  then 


K 


15. 


.10  with  an  error  less  than  (0.05)  (0,05)  = 
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B.   CRITICAL-LEVEL 

Let  6  =  max     (F  (x)  -  S  (x))  where  S  (x)  represents 

— oo<X<°° 

the  empirical  cumulative  distribution  function  of  sample 
(Fig.  1) 


Figure  1 
6~  is  the  maximum  vertical  difference  between  the 
expected  cumulative  distribution  function  and  the  observed 
cumulative  distribution  function,   Draw  horizontal  lines 
I.    with  intercept  6~+—   where  0  <  j  ,<  n(l  -6").   Then  compute 

the  value  of  f  .  =  1  -  ( 6~  +  ^-) .   For  j  =  0  the  line  £   has 
j       v      ny       J  o 

intercept  6  .   The  value  of  f .  is  f  =1-6.   For  j=l  the 

jo 

intercept  of  I,    is  (6~-f — )  and  f,  =   l-(6~+— )    etc. 
^       i     v    n'       i       v    n' 

If  the  horizontal  lines  I  .    intersects  F  (•)  at  a  jump 
point  (at  a  point  x  of  discontinuity)  the  value  of  f .  is  1 
minus  the  ordinate  F  (x)  at  the  top  of  jump. 


17 


As  shown  by  Conover  (2),  the  critical  level 


K 


>,   5 


is; 


n(l-6  ) 


[D-  *  5  ]  =  S 


(n)  f.n"Jc. 


(2) 


where 


k-1 


C.  =  1  - 


j  =  0 


(k)  f- 
J    J 


(3) 


k  is  defined  to  be  the  largest  value  of  the  subscript  j 

such  that  f .  >  0 , 
J 

Conover 's  derivation  and  proof  are  tedious  and  are 

omitted  here,   For  example,  for  k=5  equation  (3)  becomes; 

C   =  1 
o 

C.  =  1-f 
1      o 


CQ  =  1-f 
2      o 


2fici 


C3  =  l~*o   -  3flCl  "  3f2C2 

C4  =  1-f/  -  4fl3Cl  -  6f22  -  4f3C3 


To  calculate  P 


D  +  >  6  + 
n 


,  where  6   =  max 

— oo<x<°° 


ls» 


(x)  -  FQ(x)V, 


draw  horizontal  lines  t.   with  intercepts  1  -  (6   +  — )  .   The 

J  n 

value  of  f.  is  1  -  (6   +  — ) ,  but  if  the  horizontal  line 
J  ny 

I.    intersects  F  (•)  at  a  jump  x,  then  the  value  of  f.  is 
J  ov  '  J      l  J 

the  height  of  F  (x)  at  the  bottom  of  the  jump.   For  this 
critical  value,  equation  (2)  becomes: 

n(l-6+) 


Dn  >° 


1 


3=0 


(n)  f .C. 
3         3    3 


(4) 


where  c.  is  given  in  equation  (3) 
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Suppose  we  have  a  sample  of  size  n  grouped  into  k 

intervals,  having  m.  observation  for  the  i —  intervals, 

k  1 

so  Y~     ra.=n.   Let  b .  be  a  fixed  class  boundary  of  the  i — 
L=l   X  X 

or 

interval  and  Sto(«)  be  the  empirical  cumulative  distribution 

cr 

function  of  the  grouped  sample.  S  (•)  is  a  step  function 
which  rises  by  jumps  of  d.  where  0<d.<  1, 

For  example,  for  i=l,  the  jump  occurs  at  b1  with  the 

cr 

value  of  d1 .  The  first  jump  of  Sto(*)  occurs  at  the  first 
class  boundary.  For  i=2,  the  second  class  boundary  is  b~ 
and  at  this  point  the  value  of  the  step  function  is  (d-  +  dp) 

cr 

So,  the  step  function  Sto(»)  can  be  written  as: 


0 

J 


Sg(x) 
n v  y 


Z    d. 

• H  1 


i=l 

1 


for    x  <  X,1 . 

for    b  .  ^  x  <  b 
J 

for    x  >  X,  v 
(n) 


j+l 


where 


J 


the  number  of  X. . x  ^  b. 
(i)     J 


i=l 


n 


In  using  Conover's  procedure  for  K-S,  N,  let  6   be  the 
fo  y  (g)        g 

g 
maximum  vertical  distance  between  F  (•)  and  S  (•),  that  is 


6   =  Max    (F  (x)  -  Ss(x)),  or  it  can  be  written 
g      f  ov .      nv 

bk 


xe 


(Fig.  2)  as; 


6   =  Max  (F  (b. )  -  Sg(b. )) 
g       v  ov  i'     nv  i  y 
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Figure  2 
As  before,  draw  horizontal  line  I.    and  then  compute  the 

values  of  f  .  .   The  values  of  C  .  are  calculated  by  using 

J  J  y 

equation  (3)  and  finally  the  critical  level  computed  by 

equation  (4) . 

As  the  sample  size  is  increased,  the  calculation  (if 

done  by  hand)  will  be  more  tetious,  so  this  procedure  should 

be  used  for  small  sample  sizes,  say  less  than  30.   Since 

the  critical  level  is  a  decreasing  function  of  6   and  6  , 

g      g 

the  bigger  the  value  of  this  maximum  vertical  difference 

between  F  (•)  and  Sg(«)  the  smaller  will  be  the  critical 

o         n v  ' 

level,  and  vice  versa.   For  illustration  the  following  is 
an  example  of  how  to  use  Conover's  procedure.   Suppose  a 
random  sample  of  size  16  is  drawn  from  some  population  F„( • ) 
The  observed  values  are  (ordered): 
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10,35 
10,40 
10.42 
11.46 
11.49 


12,04 
12.53 
12,63 
13,18 
13,33 
13.54 


14.16 
14.26 
14,40 
14,47 
14.84 


The  hypothesis  is  Hq :  X  ~>  u  (10,20)  with  alternative 

H  :  X  r-j^  U  (10,20),   The  samples  are  grouped  into  three 

intervals,   The  values  of  the  class  boundaries  choosen 

to  be  b  =12,  b  =14  so  the  three  intervals  are:  (-°°  -12) 
1       £ 

(12  -  14)  and  (14  -  °°).  (Fig,  3) 


-Mlf 


■  Zoo 


$a     =..5*160 


?„  C*> 


tZ 


\k 


14.84 


Figure  3 


Sg(0  at  the  first  class 
n 


The  values  of  the  step  function 

boundary  is  Sg(x)  =  yg  (the  number  of  X±   <J  12)  =  .3125,  at 

the  second  class  boundary  is  Sg(x)  =  -^   (the  number  of 


n 


16 
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X.  <  14)  =  ,6875,   Under  the  null  hypothesis,  the  theoretical 

rates  for  these  three  intervals  are  — — =  .20,  — ~ 

14.84  -  10  1U  1U 

.40  ana     1Q     m      484   reSpectively . 

Here  the  maximum  vertical  difference  is  1  -  .484 

=  .5160.   The  £   with  intercept  .5160  intersects  F  (•) 

o  ov  ' 

at  jump  point  so  f   =  1  -  ,6875  =  .3125,  £   with  intercept 
,5160  +  T^  =  ,5785  also  intersects  f  (•)  at  jump  point 
so  f     .3125  and  finally  the  intercept  of  £2  is  .6410, 
so  f   =  .3125,   The  value  of  k  in  formula  (3)  is  equal  to 
2  since  £3  intersects  F  (X)  at  a  jump  point  with  top 
ordinate  1,00,  so  f  =0  which  is  not  considered,   By  using 
formula  (3)  C.  is  computed  and  found  to  be  C  =1,  C   =  ,6875, 
C2  =  .4727.   Thus,  P  [~Dn~  >,   .5160    =  (.3125)16  +  C1^) 

( .3125)1 5( .6875)  .+  ( 1| )(. 3125 ) 1  " (. 4727)  =  ,000005. 


Since  F  (x)  is  symmetric  P 


o*  L  n 


_+  >y     .5160 


is  equal  to 


the  above,  so  that  P 


D   ^  .5160 
n 


=  2  x  .000005  =  .00001. 


Using  a  =  . 01  the  null  hypothesis  should  be  rejected  since 
0.00001  <  0.01.   Incidentally  this  is  a  correct  decision 
in  this  case,  since  the  above  random  sample  was  actually 
drawn  from  a  population  having  distribution  U(10,  15). 
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IV.   COMPARISON  AND  THE  K-S,  N  TEST  POWER 

: LgJ 

The  basic  difference  among  the  three  tests,  the  Chi- 

square,  the  K-S  and  the  K-S,  *  is  that  the  Chi-square  test 

is  sensitive  to  vertical  deviation  between  the  observed 

and  expected  histogram,  the  K-S  test  is  based  on  vertical 

deviation  between  the  observed  and  expected  cumulative 

distribution  function,  whereas  K-S,  *  is  based  on  vertical 

(g) 

deviation  between  the  observed  and  expected  cumulative 
distribution  function  associated  with  discrete  groups. 
Another  obvious  difference  is  that  K-S  statistic  is  distri- 
bution free,  whereas  for  K-S,  v.  the  statistic  is  not 

(g  ) 

distribution  free. 

Both  Chi-square  and  K-S,  v  require  that  the  data  be 
grouped;  in  contrast,  the  K-S  test  does  not.   Therefore 
when  the  underlying  distribution  is  continuous  the  K-S  test 
permits  us  to  investigate  the  goodness-of-f it  with  informa- 
tion from  each  observation.   By  contrast,  both  Chi-square 

and  K-S,  %  lose  some  information  since  individual  obser- 
(g) 

vations  are  grouped  into  a  relatively  small  number  of  classes 

Further,  the  Chi-square  and  K-S,  N  tests  are  affected  by 

M  (g) 

the  number  and  the  length  of  the  class  intervals  which  are 
choosen  arbitrarily  by  the  experimenter.   The  following  is 
another  example  of  applying  the  K-S,  ,  which  is  followed 
by  comparison  with  Chi-square  test.   Suppose  we  have  a 
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random  sample  of  size  n=15,  drawn  from  some  population 
FY(#).   They  are  (ordered); 

2.01  3.91  14.89 

2.08  5.32  15.27 

2.24  9.09  21,34 

2.52  29,73 

2.70  36.54 

39.43 
40.74 
The  hypothesis  to  be  tested  is  that  S  (x)  has  come  from 
exponential  distribution  with  X=6  at  a=0.05. 

H  :  X  ~  EXP  (6) 

o  v  ' 

H,:  X^P  EXP  (6) 

Suppose  the  sample  had  been  divided  into  three  groups 

associated  with  the  intervals  (0,  2.70),  (2.70,  9.09)  and 

(9.09,  °°).   Under  the  null  hypothesis  the  expected  grouped 

c.d.f.  has  ordinates;  .3624,  .7791  and  1  respectively.   The 

observed  and  expected  frequencies  for  each  group  are 

5,  3,  7  and  5.436,  6.2505,  3.3135  respectively. 

The  maximum  vertical  difference  6  occurs  at  x  =  9.09 

g 

where  6  =  ,7791  -  .5333  =  .2458.   The  values  of  f .  and  C. 

o  J  J 

are  computed  as  in  the  previous  example  and  are  found  to  be; 
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f0  = 

.6376 

f7  = 

.2209 

co 

s 

1 

C7  = 

.1585 

fl  ■ 

.6376 

f8  " 

.2209 

ci 

= 

.3624 

C8  = 

.1287 

f2  " 

.2209 

C2 

= 

,1314 

f3  = 

.2209 

C3 

= 

.2117 

u  ■ 

.2209 

C4 

= 

.2335 

f5  ■ 

.2209 

V 

C5 

= 

.2197 

f6  " 

.2209 

C6 

= 

.1914 

With  those  values  of  f  .  and  C,  P  f"  D    >.  ,2458  I   is  found 

J      J    L  n   ^      J 

to  be  .0395.   Using  a  similar  procedure  it  can  be  seen  that 
P  [d  +  >  .2458]  =  ,0.0162,  so  P  [  D   >  .2458    is  approxi- 
mately 0.0395  +  0.0162  =  0.0557.   The  hypothesis  is 
accepted  at  a  =  0.05. 

The  critical  value  of  the  Chi-square  statistic  on  the 
same  sample,  with  three  groups  and  hence   2  degrees  of 
freedom,  is  5.3936.   This  is  less  than  5.99,  the  x2 
critical  value  f or  a =  0.05.   Again  the  hypothesis  would  be 
accepted,  that  is  both  tests  accept  that  the  random  sample 
has  been  obtained  from  a  population  having  exponential 
distribution  with  A=6.   Using  interpolation  in  tables  of 
the  incomplete  gamma  function,  the  critical  value  of  the 
Chi-square  statistic  associated  with  a=0,0557  is  found 
to  be  approximately  5.7724.   For  a=0,06  this  approximate 
value  is  5.6275.   If  we  had  used  a=0.06  rather  than  0,05 
the  K-S,  ,  test  would  have  rejected  the  null  hypothesis 
since  0.0557  <  0.06,   However  the  Chi-square  test  would 
have  accepted  the  null  hypothesis  since  5.3936  is  less 
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than  5.6275.   The  Chi-square  test  will  reject  the  null 
hypothesis  at  any  a  level  which  is  greater  than  0.06. 

In  fact,  the  random  sample  for  the  present  examples 
was  generated  from  a  population  having  an  exponential  distri- 
bution with  mean  15.   Although  the  critical  level  a=0.06 
is  not  commonly  used,  the  Chi-square  cannot  detect  that 
the  random  sample  did  not  come  from  exponential  distribution 
with  mean  6. 

From  the  example  in  Chapter  III,  it  was  found  that  the 

null  hypothesis  was  rejected  at  a=0.01.   With  the  same  data, 

with  three  groups,  hence  2  degrees  of  freedom,  under  the 

null  hypothesis  the  value  of  the  Chi-square  test  statistic 

is  13.4075,   This  is  greater  than  10.6,  the  X/  \  value  at 

\  2  ) 

a=0.05.   Thus,  both  tests  reject  the  null  hypothesis  at 
a=0.05.   The  critical  value  of  the  Chi-square  test 
associated  with  a=0. 00001  is  found  to  be  approximately 
23.4,  which  means  that  if  we  had  used  a=0. 00001,  the  Chi- 
square  test  would  have  accepted  the  null  hypothesis,  since 
13.4075  <  23.4.   Of  course,  the  critical  level  a=0. 00001 
is  rarely  used  in  practice.   The  acceptance  to  the  null 
hypothesis  by  the  Chi-square  test  at  a=0. 00001  is  most 
likely  caused  by  the  fact  of  very  low  value  of  the  expected 
frequencies  in  the  fourth  group  of  the  sample  that  is  1,344. 

Suppose  the  null  hypothesis  is  H  :  X/^U(10,  18)  with 
critical  level  a=0,05.   Under  this  null  hypothesis,  the 
value  of  the  Chi-square  statistic  is  5.320,  which  is  less 
than  10.6.   In  this  case,  the  Chi-square  test  accepts  the 
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null  hypothesis,  or  in  other  words,  the  Chi-square  test 

accept  that  the  random  sample  was  generated  from  a 

population  having  U(10,  18)  distribution.   Again  in  this 

case  the  Chi-square  test  has  low  expected  frequencies  in 

the  fourth  group  namely  1,92. 

For  the  K-S,  x  test,  under  the  null  hypothesis 
(g) 

U(10,  18),  the  value  of  6   is  .395,  so  P  [d   >,  ,395   = 

0.00008,  which  is  less  than  0.05.   The  K-S,  N  test  rejects 

(g) 

the  null  hypothesis,  which  means  rejects  that  the  random 
sample  was  generated  from  a  population  having  U(10,  18) 
distribution. 

The  present  examples  suggest  the  K-S.  .  may  be  more 
powerful  than  the  Chi-square  test,  at  least  in  certain 

discussed  below.  -^ 

In  the  previous  example,  concerning  the  exponential 

distribution,  the  critical  value  of  the  K-S  statistic  in 

the  continuous  case  at  a=0.0557  is  between  .304  and  .338, 

which  leads  us  to  reject  the  null  hypothesis  that  the 

sample  was  generated  from  an  EXP  (A=6)  distribution.   At 

the  same  critical  level,  for  example  0.0557  or  0.00008, 

grouping  samples  into  intervals  tends  to  lower  the  power, 

Thus,  for  grouped  data  (samples),  the  appropriate  critical 

values  are  smaller  than  those  tabulated  for  continuous 

case.   Thus,  use  of  tables  of  critical  values  of  the  K-S 

test  will  not  give  a  correct  test.   Therefore  the  need 

for  the  K-S.  .  procedure  is  evident, 
(g) 
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V,   SIMULATION 

In  order  to  get  a  better  comparison  between  the  three 
goodness-of-f it  tests  previously  mentioned,  especially 
their  powers,  simulation  was  used.   The  powers  of  the  tests 
should  be  compared  under  the  same  conditions,  namely  at 
the  same  significance  level  and  for  the  same  null  hypothesis. 
There  are  two  procedures  for  this  simulation,   The  first 
procedure  is  as  follows:   generate  a  random  sample  of  size 
n  from  a  population  having  distribution  F  (•)•   Specify 
a  critical  level  a,    for  example  a   and  find  the  critical 

value  6   of  the  K-S  statistic  associated  with  a1    and  n. 

By  using  tables  of  the  incomplete  gamma  function,  one 

can  calculate  (at  least  approximately)  the  critical  values 

of  the  Chi-square  statistics  associated  with  a  .   Let  C. 

denote  this  critical  value.   These  three  values,  a    ,  6  ,  Cx , 

are  then  used  as  the  input  to  the  simulation  for  computing 

the  relative  frequency  of  acceptance  of  H  by  the  K-S.  . , 

X  D  J 

the  K-S  for  continuous  distribution  and  the  Chi-square  tests, 
respectively.   Further,  generate  the  random  samples  N  times. 
The  hypothesis  to  be  tested  is:   HQ :   Fx(*)  =  FQ( * )  and 
H,:   F  (•)=¥=  F  (•)  at  level  a,  . 

AAA 

Let  p  ,  p2 ,  p3  denote  the  relative  frequency  of 
acceptance  of  H  by  the  K-S,  the  K-S.  .  and  the  Chi-square 
respectively,  then 
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s\  "number   of    times"    (D     ^    6    ) 

v    n         .  1  ' 

p,  = n — 

"number   of   times"      P   [d      ^6,     \>,  a 
P2    =   N 


p,  = 


"number  of  times"  (x2/  n  Statistic  ^   C    ) 
(r ) ' 


3  N 

Both  acceptances  of  K-S  and  Chi-square  test  can  be  programmed. 
However,  it  appears  to  be  difficult  to  program  Conover's 
procedure.   To  the  knowledge  of  the  author,  a  subroutine 
for  this  procedure  is  not  available. 

Therefore  the  simulation  has  been  done  indirectly, 
using  a  second  procedure,   This  can  be  described  as  follows: 
With  the  same  random  samples  from  the  same  F  ( • )  and  the 
same  number  of  groups  then  for  0  <  d.  <  d.  <  1, 

P  |d   >,  d  1  >,  P  I  D   >,  d2   .   If  the  hypothesis  is  accepted 

at  P  ID   5-  d  1  =  a2  ,  then  it  will  be  accepted  at 

P  |D   >  dl        =   a    ,  since  o.  1    %,  a    ,   In  other  words  the  sample 

rates  of  acceptance  of  K-S,  .  is  defined  to  be 

(g) 

"number  of  times"  (D  -$  d) 

n 
rj where  d  is  specified  in 

advance.   Once  d   is  fixed,-  P  [  D  >,   d   =  al    can  be  computed 
from  which  the  critical  values  for  the  other  two  tests 
associated  with  a.  l    can  be  calculated.   However,  it  will  be 
difficult  to  obtain  an  approximate  value  for  the  critical 
values  of  the  K-S  test  because  of  the  limited  a  values  in 
the  tables.   Therefore,  d  is  used  to  calculate  the  frequency 
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of  acceptance  by  both  K-S  tests.   So,  the  inputs  for  the 
second  procedure  are  Cx    and  d. 

For  the  purpose  of  this  simulation,  groups  of  15  random 
numbers  were  generated  from  Exponential  distributions  with 
means  X=6,  9,  12  and  15;  random  samples  of  size  14  were 
generated  from  Normal  distributions  with  y=0,  a=l,  3  and 
5;  random  samples  of  size  16  were  generated  from  Uniform 
distributions  U(10,  13),  U(10,  15)  and  U(10,  17).   The 
value  of  d  was  choosen  arbitraryly  to  be  d= . 3 .   The  results 
of  this  simulation  are  summarized  in  table  (5.1). 

The  null  hypothesis  for  the  four  Exponential  cases  was 
F   =  EXP  (X=9),  with  4  groups  given  by  the  intervals  (0  -3), 
(3-6),  (6  -  9)  and  (9  -  100).   By  Conover's  procedure 
P  I'D   £  d=.3j  =  a®  =  0.0328.   The  null  hypothesis  for  the 
three  Normal  cases  was  N(0,  1),   Four  groups  were  used 
with  intervals  (-°°,  -1),  (-1,  0),  (0,  1)  and  (1,  °°)  and 
P  ["D   >  d=.3~  =  aj  was  found  to  be  0.0185.   Finally,  the 
null  hypothesis  for  the  Uniform  cases  was  U910,  15),  with 
5  groups  given  by  the  intervals  (10,  11),  (11,  12),  (12,  13), 
(13,  14)  ,  (14,  17)  and  P  [ Dr  >  d= . 3    =   a\    was  found  to 
be  0.022.   The  associated  critical  values  for  the  Chi- 
square  test  are  approximately  8.7517,  8.00,  13.245  in  the 
EXPONENTIAL,  NORMAL  and  UNIFORM  cases,  respectively.   These 
three  critical  values  together  with  the  three  a  levels  are 
then  used  in  the  input  to  the  simulation. 

Simulation  results  give  p2 =97.97%  for  the  acceptance 

frequency  of  the  K-S,  x  test  for  the  null  F   =  U(10,  15) 

(g)  o 
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distribution,  which  mean  (1,00  -  .9797)  =  0,0203  is  the 
estimated  rejection  rate,  in  other  words  a  =0.0203,   The 
input  a.  for  the  Uniform  random  cases  is  0.022;  the  differ- 
ence 0.0017  between  these  is  not  significant,   From  table 
(5.1)  it  can  be  seen  that  the  corresponding  difference  for 
the  Exponential  case  is  0,0028  and  for  the  Normal  case  is 
0.0008,  which  are  not  significant,   From  the  same  table  it 
can  be  seen  that  the  significance  level  (for  the  null 

hypothesis) for  both  the  Chi-square  and  K-S,  N  tests  in 
'*  (g) 

the  Uniform  and  Exponential  cases,  are  very  close. 

For  example,  in  the  Exponential  case,  this  difference 
is  only  (.0356  -  .0316)  =  ,004.   Therefore  these  tests  can 
be  assumed  to  be  at  the  same   a  level.   In  the  Normal  case 
by  grouping  the  data,  the  value  of  0^=0.0177.   This  gives 
a  difference  of  0.0008  compared  with  the  input  a  =0.0185, 
However,  the  critical  value  of  the  Chi-square  statistic 
associated  with  a ^0.0185,  and  degrees  of  freedom  3  is  8,0 
which  gives  .9565  for  the  acceptance  rate  under  null 
hypothesis.   In  other  words  the  a  value  here  is  0,0435, 
which  is  not  sufficiently  close  to  0.0185  since  the  differ- 
ence is  0.0250.   To  get  a  closer  value,  the  critical  value 
of  8.0  was  adjusted  to  10.50  which  gave  a^O.0126.   This 
adjustment  causes  the  other  two  acceptance  rates  for  N(0,  9) 
and  N(0,  25),  to  be  larger. 

With  this  adjustment  all  the  tests,  Chi-square,  K-S 

and  K-S,  N  are  at  (nearly)  the  same  a  level,  so  comparison 
(g)  " 

of  their  powers  are  easily  performed. 
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For  the  U(10,  17)  case  the  power  of  the  Chir-square 

and  the  K-S.  .  are  0.2208  and  0,2426  and  for  U(10,  13)  both 

are  equal  to  1.00.   It  means  that  in  the  U(10,  17)  case 

the  K-S,  N  test  appears  to  be  more  powerful  than  the  Chi- 
(g) 

square  test,  whereas  in  the  U(10,  13)  case  both  tests  have 

the  same  power.   For  the  Exponential  case  with  parameter 

6,  12  and  15  the  power  of  the  K-S,  N  test  turns  out  to 

(g) 

be  more  powerful  than  the  Chi-square  test  in  these 
Exponential  and  Normal  cases.   Looking  at  the  results  of 
another  simulation,  using  three  samples,  generated  from 
U(0,  18),  U(0,  20)  and  U(0,  22)  distributions,  where  the 
null  hypothesis  to  be  tested  was  the  Exponential  distribu- 
tion with  mean  A=9,  which  are  also  tabulated  in  table  (5,1), 

the  K-S,  N  appears  to  be  more  powerful  than  the  Chi-square 
(g) 

test . 

For  the  Normal  case,  the  Chi-square  test  turns  out  to 
be  more  powerful  than  the  K-S ,    . ,  even  though  the  expected 
frequencies  in  the  first  and  fourth  groups  were  very  low, 
only  2.220  and  2.218.   Of  course,  the  Chi-square  distribu- 
tion in  these  applications  is  only  an  approximation.   This 
approximation  gives  better  result  for  larger  samples.   It 
is  a  rule  of  thumb  that  the  approximation  can  be  used  with 
confidence  as  long  as  every  expected  frequency  is  at  least 
equal  to    5.   This  rule  should  not  be  considered  inflexible 
however.   It  appears  to  be  a  conservative  value  and  the 
Chi-square  approximation  was  reasonably  accurate  in  this 
study,  even  for  expected  cell  frequencies  as  small  as  1.5 
(6). 
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Based  on  the  results  of  the  simulation  for  Exponential 
cases  where  two  of  the  four  groups  have  low  expected 
frequencies  (namely  3.0315  and  2.2350),  the  approximation 
appears  reasonably  accurate.   The  estimated  value  of  a, 
obtained  from  this  simulation  is  0^=0.0316  which  is  close 
to  the  "target,"  0^=0.0328.   By  contrast,  for  the  Normal 
case,  the  approximation  is  not  so  accurate,   Thus,  it  was 
adjusted  to  get  a  closer  value,  as  mentioned  before.   This 
inaccurate  approximation  may  be  caused  by  the  fact  of  small 
expected  frequencies  in  the  tail. 

Table  (5,2)  shows  results  of  yet  another  simulation. 
By  keeping  the  variance  constant  and  letting  the  means  vary, 
again  the  K-S,  .  test  appears  to  be  more  powerful  than 
the  Chi-square  test. 

In  all  cases,  without  grouping  the  data,  in  other  words, 

in  the  continuous  case,  the  ordinary  K-S  test  is  the  most 

powerful  test.   Existing  tables  for  the  K-S  distribution 

indicate  the  critical  value  6=.3  corresponds  to  an  a  level 

somewhere  between  .10  and  .15.   To  get  the  same,  or  at 

least  close  to  the  critical  level  used  for  the  K-S,  x,  for 

(g) 

example  0.0328  for  the  Exponential  case,  d= . 3  should  be 
increased.   In  other  words,  the  value  of  6  for  continuous 
case  would  be  greater.   Again,  grouping  the  data  and 
holding  the  same  a  level  requires  the  critical  value  of  the 
K-S  test  to  be  smaller  than  that  tabulated. 
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VI.   SUMMARY 

1.  Based  on  the  results  of  the  simulations,  grouping  the 
data  causes  the  K-S  statistic  to  be  stochastically 
smaller  than  that  for  the  ordinary  tabulated  K-S 
statistic.   For  continuous  underlying  distribution 
functions  the  K-S  test  is  the  most  powerful  of  the 
three  test  considered. 

2.  It  is  hard  to  draw  a  general  conclusion  related  to 

the  relative  powers  of  the  K-S,  .  and  Chi-square 

tests.   In  some  cases  the  K-S,  N  test  is  more  powerful 

(g)  y 

than  the  Chi-square  test;  in  other  cases  the  reverse 
is  true.   It  is  su^^ested  th?.t  the  rela.tive  powers  be 
investigated  in  more  detail.   Further  investigation  is 
also  needed  to  determine  rules  of  thumb  for  the 
appropriate  number  of  groups  to  be  used. 

3.  For  small  samples,  the  ordinary  table  of  the  K-S 
statistic  cannot  be  used  for  grouped  data.   The  extension 
of  the  K-S  test  suggested  herein,  should  be  used  when 
the  data  have  been  grouped.   Unfortunately,  for  sample 
sizes  larger  than  30  the  calculation  becomes  tedious. 

4.  It  would  be  very  worthwhile  to  program  a  subroutine  of 
Conover's  procedure.   The  availability  of  the  subroutine 
would  enable  this  test  to  be  used  without  time  consuming 
calculations,  and  also  possibly  make  the  test  available 
for  larger  sample  sizes. 
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5.  It  is  suggested  that  the  possibility  of  developing  a 
"quick  and  dirty"  modification  of  the  K-S  table  for  use 
in  discrete  and  grouped  cases,  be  investigated. 

6.  This  simulation  also  demonstrated  the  adequacy  of  x2 
approximation,  even  when  expected  cell  frequencies 
fell  substantially  below  5. 

7.  Care  should  be  taken  in  computing  D  ,  the  maximum 

vertical  distance  between  F  (x)  and  Sg(x).   This 

o   '      nv 

maximum  distance  is  equal  to  the  greatest  value  of 


F  (b. )  -  S&(b. ) 
o   l      n v  l 


where  b.  ranges  over  the  set  of 


class  boundaries. 
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